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ABSTRACT 

Standards  and  Benchmark  Tests 

for  Evaluating  Large  Scale  Manipulators 

with  Construction  Applications 

by 

Mark  Edward  Wiersma,  M.S.E. 

The  University  of  Texas  at  Austin,  1995 

COSUPER VISOR:  Alfred  E.  Traver 

COSUPERVISOR:  Carl  T.  Haas 

New  applications  of  large  scale  manipulators  for  construction  are 
continually  emerging  and  making  them  multi-functional  construction  machines. 
This  report  presents  a  set  of  meaningful  benchmark  tests  to  gauge  the  overall  static 
and  dynamic  performance  of  different  large  scale  manipulators  in  order  to  achieve 
a  means  of  relative  comparison.  The  most  important  static  and  dynamic 
performance  criteria  are  defined  and  a  method  provided  for  evaluating  them. 

The  report  reviews  the  application  of  human  factors  engineering  to  large 
scale  manipulator  acquired  by  The  University  of  Texas.  Furthermore,  a  method  is 
presented  to  quantify  potential  improvements  to  the  human-machine  interface. 
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CHAPTER  ONE 
INTRODUCTION 

1.1  Background 

The  construction  industry  is  ready  for  the  introduction  of  autonomous 
technology.  In  particular,  large  scale  manipulators,  for  instance  automated  back 
hoes  and  rebar  placement  machines,  have  shown  great  promise  in  improving 
safety,  productivity,  quality  of  construction  and  environmental  impact  (Hsieh  and 
Haas  1993).  The  Construction  Automation  Group  at  the  University  of  Texas 
focuses  on  the  development  of  such  systems. 

1.1.1  Grove  Pipe  Manipulator 

The  large  scale  manipulator,  hereafter  referred  to  as  the  Manipulator, 
owned  by  the  University  of  Texas  at  Austin,  was  originally  designed  by  DuPont  as 
a  crane  attachment  for  the  purpose  of  installing  large  bore  piping.  The  designs 
were  submitted  to  the  Grove  Manufacturing  Company  for  construction  and  the 
"Pipe  Manipulator"  became  a  reality  in  1980.  Glass  (Glass  1984)  presents  a 
complete  description  of  the  crane  mounted,  electrohydraulically  controlled  device. 

The  initial  expectations  for  the  Manipulator  were  high.  It  was  envisioned 
that  the  Manipulator  would  replace  the  smaller,  but  more  versatile,  "cherry  picker" 


(15-ton  crane)  in  the  erection  of  large  piping  systems  like  those  found  at  large 
chemical  processing  plants.  After  its  debut,  however,  the  Manipulator  proved  to 
be  slow,  difficult  to  control  and  uneconomical  to  operate.  As  a  result,  workers 
regarded  the  Manipulator  as  inferior  to  the  "cherry  picker"  and  left  it  abandoned 
on  the  job-site.  Supporters  of  the  Manipulator  were  reluctant  to  give  up  on  their 
new  apparatus  and  thus  invited  the  University  of  Texas  to  conduct  a  study  to 
assess  its  merits  and  deficiencies.  Figure  1 . 1  shows  the  Manipulator  mounted  on  a 
22  ton  crane. 


Figure  1.1  -  Crane-Mounted  Pipe  Manipulator 


1.1.2  Previous  Manipulator  Testing 

Glass  (Glass  1984)  was  the  first  to  conduct  a  complete  assessment  to 
ascertain  the  Manipulator's  strengths  and  weaknesses.  Conducted  on  the 
construction  site  of  a  new  chemical  plant  in  southeast  Texas,  Glass's  investigations 
determined  that  the  Manipulator  was  plagued  with  numerous  design  flaws  which 
hindered  its  mobility,  speed  and  overall  economic  performance  when  compared  to 
the  capabilities  of  the  "cherry  picker".  He  further  concluded  that  the  control 
system  was  a  primary  contributing  factor  by  reason  of  its  number  of  control  levers 
and  their  location,  an  operator  basket  fastened  directly  to  the  machine.   Since  the 
gravity  leveling  operator  basket  was  mounted  together  with  the  Manipulator,  slow 
arm  movements  were  imperative  for  providing  a  safe  ride  for  the  operator.  This 
suggests  that  the  Manipulator  was  inherently  slow  for  a  reason.  In  addition,  the 
eight  control  levers,  necessary  for  controlling  the  Manipulator's  eight  degrees-of- 
freedom,  often  left  the  operator  hesitating  to  determine  proper  maneuvering 
sequences  to  optimize  performance.  Amongst  Glass's  conclusions  are  the 
recommendations  for  removing  the  operator  basket  from  the  support  frame  and 
replacing  the  existing  control  levers  with  "joy-stick"  controls.  Future  research  and 
design  would  heed  these  recommendations. 

Hughes  (Hughes  1990)  hypothesized  "that  the  Manipulator  will  compete 
with  a  conventional  crane  in  pipe  spool  erection  cost,  by  use  of  a  simple  improved 


teleoperator  interface  for  arm  control".  Hughes'  research  culminated  in  the 
development  of  an  entirely  new  man-machine  interface  with  ergosticks  serving  as 
the  controller.  In  designing  a  teleoperated  controller,  he  also  removed  the  controls 
from  the  operator  basket  allowing  the  operator  to  select  his  position  anywhere 
within  the  range  of  a  tether  of  control  cables  (approximately  length  100  ft). 

To  validate  his  hypothesis,  Hughes  contrived  a  test  scheme  fundamentally 
similar  to  the  one  used  by  Glass.  Although  the  test  was  a  simple  pick-and-place 
scenario  (Fisher  1989),  it  represented  the  first  attempt  at  creating  a  benchmark  to 
compare  control  schemes.  Since  his  tests  were  performed  in  a  laydown  yard  with 
inexperienced  operators,  Hughes  compared  his  data  to  Glass'  only  after  applying  a 
"time  productivity  transformation  equation"  (Hughes  1990,  pp.  120).  This 
equation  corrected: 

1)  The  simplification  of  the  pipe  rack  compared  to  the  chemical  plant 
configuration; 

2)  The  inexperience  of  Hughes'  volunteers; 

3)  The  remote  location  of  the  control  box  removed  from  the  operator 
basket. 

In  essence,  Hughes  used  Glass'  statistics  as  a  benchmark  to  conclude  whether  his 
ergostick  interface  controller  was  an  improvement  over  Groves'  levers.  He 
concluded  it  was  not. 


Thomas  (Thomas  1995)  continued  to  seek  a  more  advanced  controller  that 
would  increase  the  Manipulator's  productivity.  His  research  concluded  with  the 
purchase  of  a  six  degree-of-freedom  optical  force  and  torque  sensor  Dimension  6 
Geometry  Ball  manufactured  by  CIS  Graphics,  Inc.  The  new  controller  was 
installed  and  tested  for  functionality.  Up  to  this  time  no  formal  benchmark  tests 
have  been  performed. 

In  addition  to  his  controller  research,  Thomas  moved  the  Manipulator  into 
the  controlled  confines  of  the  Construction  Automation  Lab.  His  design  of  a 
"cantilevered  space  frame"  as  a  new  Manipulator  mount  has  made  future 
development  a  more  efficient  and  pleasant  task.  It  is  important  to  notice,  however, 
that  removing  the  Manipulator  from  the  crane  mount  reduces  the  number  of 
degrees-of-freedom  from  eight  to  six.  Lab  tests  will  be  performed  without  the 
extendibility  of  the  crane,  thus  reducing  the  scope  of  the  overall  analysis.  Figure 
1.2  shows  the  new  Manipulator  configuration. 


Figure  1.2  -  Manipulator  Mounted  on  Cantilevered  Space  Frame 


1.2  Research  Objectives 

New  applications  of  large  scale  manipulators  for  construction  are 
continually  emerging  and  making  them  multi-functional  construction  machines. 
The  objective  of  this  report  is  to  provide  a  set  of  meaningful  benchmark  tests  to 


gauge  the  overall  static  and  dynamic  performance  of  different  large  scale 
manipulators  in  order  to  provide  a  means  of  relative  comparison.  The  areas  of 
performance  measures,  human  factors,  statistical  analysis  and  benchmark  tests  for 
large  scale  manipulators  are  investigated. 

1.2.1  Performance  Measures 

Today,  a  large  variety  of  Manipulators  are  employed  for  many  different 
uses  ranging  from  the  manufacturing  assembly  line  to  the  construction  site. 
Because  such  a  vast  assortment  of  Manipulators  with  diverse  uses  and  shapes 
exists,  standardizing  specifications  is  difficult.  However,  there  are  certain 
qualifications  which,  all  else  being  equal,  permit  Manipulators  of  similar  type,  size 
and  function  to  be  compared.  This  report  reviews  various  methods  of 
performance  qualification  for  large  scale  manipulators  for  the  purpose  of 
ascertaining  physical  specifications. 

1.2.2  Human  Influence  Analysis 

A  common  paradox  encountered  in  advancing  control  technology  is  the 
development  of  new  systems  that  can  potentially  overwhelm  their  human 
operators.  Since  the  human  operator  plays  a  vital  role  in  the  performance  of  the 
Manipulator,  human  factors  cannot  be  over  looked.  System  efficiency  will  remain 


as  dependent  upon  the  capabilities  of  the  operator  as  it  does  upon  the  capabilities 
of  the  Manipulator  and  its  human-machine  interface.  The  application  of  human 
factors  engineering  to  the  Manipulator  is  examined.  In  addition,  a  method  to 
quantify  potential  improvements  to  the  human-machine  interface  is  presented. 

1.2.3  Applying  Statistical  Analysis 

This  report  examines  the  statistical  methods  for  determining  the  probability 
that  the  Manipulator  will  achieve  a  desired  target  within  a  calculated  standard 
deviation.  The  formulas  for  computing  position  accuracy,  repeatability  and  there 
respective  standard  deviations  are  reviewed  for  both  static  and  path-related  output. 

1.2.4  Benchmark  Tests  for  the  Large  Scale  Manipulator 

This  report  will  define  the  benchmark  test  procedures  for  evaluating  the 
performance  criteria  of  large  scale  manipulators.  The  performance  tests  presented 
in  this  report  are  based  on  the  American  National  Standard  for  Point-to-Point 
(Static)  Performance  and  for  Path-Related  (Dynamic)  Performance  evaluations  for 
industrial  robots  and  robot  systems.  The  performance  criteria  are  accuracy, 
repeatability,  cycle  time,  overshoot,  settling  time,  relative  path  accuracy,  path 
repeatability,  path  speed  characteristics  and  cornering  overshoot.  Methods  of 
measuring  the  performance  criteria  are  not  discussed  in  this  report. 


CHAPTER  TWO 
PERFORMANCE  MEASURES 

This  chapter  reviews  various  methods  of  performance  qualification  for 
large  scale  construction  Manipulators  for  the  purpose  of  ascertaining  physical 
specifications.  There  are  several  reasons  for  performance  specifications: 

•  selecting  a  machine  for  a  given  task, 

•  planning  work  so  that  tasks  fall  within  the  capabilities  of  a  given 
machine; 

•  setting  goals  for  new  equipment; 

•  generating  benchmarks  to  gauge  performance  parameters  of  different 
machines,  systems  and  techniques. 

Today,  a  large  variety  of  Manipulators  are  employed  for  many  different 
uses  ranging  from  the  manufacturing  assembly  line  to  the  construction  site. 
Because  such  a  vast  assortment  of  Manipulators  with  diverse  uses  and  shapes 
exists,  standardizing  specifications  is  difficult.  However,  there  are  certain 
qualifications  which,  all  else  being  equal,  permit  Manipulators  of  similar  type,  size 
and  function  to  be  compared.  The  following  sections  define  these  qualifications. 


2.1  Static  vs.  Dynamic  Outputs 

Output  is  the  response  of  the  Manipulator  to  input  commands. 
Output  is  a  specified  movement  or  application  of  force  to  an  object,  and  can  be 
either  static  or  dynamic  in  nature. 

Static  output  results  when  the  Manipulator  approaches  a  target  point  and  is 
held  in  a  fixed  position.  Rebar  and  pipe  placement  are  just  two  examples  of  static 
output.  The  important  performance  measure  is  the  final  position  of  the  end- 
effector  and  its  payload.  In  the  absence  of  obstruction,  the  path  followed  by  the 
end-effector  in  the  performance  of  the  task  is  irrelevant.  Static  outputs  are  much 
easier  to  measure  since  the  outcome  is  not  in  motion. 

Dynamic  output,  in  contrast,  results  when  the  Manipulator  follows  a 
specified  path.  Evaluation  of  dynamic  output  is  more  difficult  because  it  entails 
continuous  measurement  of  the  end-effector  position  during  the  execution  phase 
Tracking  and  surface  following  are  two  examples  of  dynamic  outputs. 

2.2  Accuracy 

Accuracy  is  the  measure  of  the  difference  between  the  desired  output  and 
the  achieved  output  when  there  is  no  memory  of  previously  performed  tasks.  The 
expected  error  of  the  output  about  the  mean  achieved  output  when  added  to  the 
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mean  accuracy  represents  some  degree  of  confidence  of  the  accuracy  (Colson 
1984). 

There  are  two  types  of  accuracy,  absolute  and  relative.  They  differ  only  in 
their  frames  of  reference.  Absolute  accuracy  is  measured  relative  to  the 
Manipulator's  base  coordinate  system.  Figure  2. 1  illustrates  how  absolute 
accuracy  relates  to  the  Manipulator's  base  coordinate  system. 


m 


Absolute  Accuracy 


Desired 
Output 


Base  Coordinate 
System 


Figure  2.1  -  Absolute  Accuracy 

Relative  accuracy  is  measured  from  a  previously  achieved  output    Relative 
accuracy  occurs  when  a  specified  output  is  planned  from  some  calibration  point 
other  than  the  base  coordinate  system.  It  is  an  especially  important  performance 
measure  when  working  with  tool  jigs  or  when  working  from  a  benchmark  on  a 
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construction  site.  It  is  assumed  that  the  relative  accuracy  of  the  Manipulator  is 
adequate  enough  so  that  given  a  calibration  point  other  than  the  base  coordinate 
system,  subsequent  output  can  be  determined  with  some  level  of  confidence. 
Relative  accuracy  is  the  measure  of  accuracy  of  location  points  within  the 
Manipulator's  work  space  to  a  calibration  point  located  within  the  work  space. 
Figure  2.2  illustrates  the  concept  of  relative  accuracy. 


Achieved 
Output 


Desired 
Output 


Figure  2.2  -  Relative  Accuracy 

2.3  Repeatability 

Repeatability  is  the  measure  of  how  closely  the  achieved  output  clusters 
about  its  mean.  Todd  (Todd  1986)  illustrates  the  difference  between  repeatability 
and  accuracy  in  the  Figure  2.3. 
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In  the  target  analogy  of  Figure  2.3,  each  dot  represents  an  attempt  to 
achieve  the  desired  output,  in  this  case,  hitting  the  cross-hairs  on  the  target.  The 
size  of  the  dot  cluster  represents  the  measure  of  repeatability  and  the  closeness  of 
the  center  of  the  cluster  represents  the  accuracy.  It  is  clear  from  this  figure  that  it 
is  possible  to  obtain  a  high  level  of  repeatability  without  being  accurate.  It  is 
important  to  note  the  difference  between  accuracy  and  repeatability. 


low  accuracy, 
low  repeatability 


low  accuracy, 
high  repeatability 


high  accuracy, 
low  repeatability 


high  accuracy, 
high  repeatability 


Figure  2.3  -  Accuracy  vs.  Repeatability  (Todd  1986) 
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Repeatability  is  a  very  powerful  benefit  to  an  autonomous  Manipulator.  It 
denotes  the  ability  to  repeat  programmed  outputs  consistently.  Repeatability  is 
more  valuable  than  accuracy;  if  the  error  is  constant,  accuracy  can  always  be 
corrected.  For  these  reasons,  repeatability  is  often  considered  more  meaningful 
performance  measure  than  accuracy. 

2.4  Resolution 

The  capability  of  feedback  devices  (encoders)  in  coordination  with  the 
control  system  in  determining  the  locality  of  the  end-effector  and  its  calibration 
point  determines  the  resolution  of  the  Manipulator.  In  broader  terms,  Wodinski 
(Wodinski  1987)  defines  resolution  as  the  measure  of  the  smallest  possible 
increment  of  change  in  the  variable  output  of  a  device  or  position  sensors. 

Although  resolution  is  ultimately  determined  by  the  capabilities  of  the 
system's  actuators  and  components,  to  the  user  it  is  simply  the  minimum 
consistently  commandable  output  that  is  measurable  at  the  end-effector. 
Resolution  comprises  both  position  and  orientation  of  the  Manipulator's  end- 
effector. 

Resolution  has  some  degree  of  impact  on  the  teach-and-repeat  capabilities 
of  the  Manipulator,  such  as  teaching  the  Manipulator  to  place  a  cylindrical  bar  into 
a  round  hole.  If  the  tolerance  of  the  bar  and  hole  fit  is  smaller  than  the  resolution 
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of  the  Manipulator  system,  then  insertion  of  the  bar  into  the  hole  may  never  be 
possible.  Fortunately  there  are  many  construction  applications  for  which  high 
tolerance  specifications  are  not  required. 

2.5  Celerity 

There  are  two  primary  performance  criteria  associated  with  the  speed  of 
the  Manipulator.  One  is  economic  and  the  other  kinematic.  Performance 
specifications  provided  by  the  manufacturer  almost  always  include  speed.  The 
engineer  embarked  on  the  advancement  of  the  Manipulator's  capabilities  must 
often  evaluate  the  machine's  swiftness  and  speed  in  economic  terms.  Engelberger 
(Engelberger,  1980),  a  pioneer  in  robotics  research,  writes  "no  matter  what  the 
social  benefits  are,  no  matter  how  clever  the  technology,  no  matter  how  pretty  the 
robot  is  to  watch,  every  proposed  investment  in  robotics  has  to  pass  the  test  of 
critical  financial  appraisal."  Perhaps  financial  performance  is  the  toughest  test  of 
all.  The  most  brilliant  innovations  are  failures  if  they  lose  money  or  even  if  they 
cannot  provide  an  attractive  return  for  the  investor.  In  theory,  at  least  all  money  is 
competing  for  the  highest  possible  return. 

Glass  (Glass  1984)  measured  the  performance  of  the  prototype  Grove 
"Pipe  Manipulator"  by  comparing  its  operating  cost  efficiency  to  that  of  the 
incumbent  "cherry  picker."  Unfortunately  for  Grove  and  DuPont,  the  "cherry 
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picker"  was  the  better  performer  and  the  Manipulator  was  taken  off  the  jobsite  and 
given  to  the  University  for  further  research  and  development. 

Hughes  (Hughes  1989),  after  devising  a  new  control  scheme,  proceeded 
with  the  same  economic  analysis  methods.  He  knew  that  replication  of  the  original 
test  procedure  was  essential  for  comparison  and  ultimately  to  validate  his 
hypothesis. 

In  order  to  properly  evaluate  the  Manipulator's  improved  control 
characteristics,  it  is  essential  to  relate  new  benchmark  tests  to  the  previously  used 
economic  ones.  This  is  the  only  way  to  quantify  performance  improvements. 
Hughes  accomplished  this  by  applying  a  "productivity  transformation  equation"  to 
his  results  to  enable  a  comparison  of  the  present  and  past.  Likewise,  such 
transformations  offer  the  opportunity  to  arrive  at  the  wrong  conclusion  if  not  done 
carefully. 

From  a  kinematic  standpoint,  both  velocity  and  acceleration  will  have 

important  impacts  upon  the  rate  of  performance  of  the  Manipulator.  Because 

cycle  time  is  of  substantial  concern  for  the  applications  engineer,  benchmarks  for 

the  maximum  speed  of  each  degree-of- freedom  will  be  important.  Appleton  and 

Williams  (Appleton  and  Williams  1987)  suggest  the  following: 

One  approach  that  is  useful  for  comparison  purposes  is  to  define  a 
test  cycle  and  measure  the  total  travel  time.  The  test  cycle  should  be 
repeated  a  number  of  times  and  a  mean  and  standard  deviation  determined 
for  the  cycle  time.  Within  a  simple  trajectory  it  might  also  be  useful  to 
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know  the  speed  variation,  the  mean  speed  and  the  average  speed,  all  of 
which  are  useful  for  building  up  theoretically  predicted  cycle  times 

Maximum  and  minimum  accelerations  will  be  important  for  end-effector 
design.  This  information  is  necessary  for  determining  the  forces  required  to  hold 
items  securely  to  avoid  slippage  during  handling  operations. 

Finally,  it  is  important  to  bear  in  mind  that  the  speed  of  the  Manipulator 
during  unconstrained  point  to  point  movement  will  be  faster  than  during 
continuous  path  control  movements  necessary  during  tracking  and  surface 
following.  The  control  of  speed  and  the  control  of  position  are  intimately  related 
for  path  following  applications. 

2.6  Overshoot  and  Settling  Time 

In  robotics  ANSI  defines  overshoot  as  the  "largest  distance  of  overtravel 
past  the  target  position  along  the  direction  of  motion  after  the  robot  is  within  a 
settling  bandwidth  (±S)."  S  is  the  standard  deviation  and  is  defined  later. 
Overshoot  is  predominate  during  "violent  changes  in  direction  and  mass  and 
during  acceleration  and  deceleration."  (Warnecke  et  al.  1985)  Cornering 
overshoot  is  defined  by  ANSI  as  the  largest  deviation  outside  of  the  reference  path 
after  the  Manipulator  has  passed"  the  corner. 
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Settling  time  is  "a  period  of  time  required  for  the  robot  to  remain  within  a 
limit  (±S)  from  the  target  point  after  a  move  command  is  executed.  Sometimes 
settling  time  is  called  oscillation.  Settling  time  is  measured  as  the  elapsed  time 
starting  from  initial  crossing  into  the  limit  band  to  the  last  point  that  is  outside  this 
limit"  Overshoot  and  settling  time  are  shown  in  Figure  2.4. 
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Figure  2.4  -  Overshoot  and  Settling  Time  (ANSI/RIA  Rl 5.05-1  90) 

Overshoot  should  be  measured  to  quantify  the  Manipulator's  capability  to 
make  smooth  and  accurate  stops  at  target  points.  This  is  an  important 
performance  measure  during  applications  involving  large  inertias,  high  speeds  or 
frequent  stops.   Settling  time  should  be  measured  to  quantify  how  quickly  the 
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Manipulator  can  stop  at  a  target  point    Overshoot  and  settling  time  are  related 
parameters.  Note  both  overshoot  and  settling  time  depend  not  only  on  the  mass 
distribution  of  the  Manipulator  but  also  on  the  mass  distribution  of  the  payload. 

ANSI/RIA  15.01-1  90  outlines  the  American  National  Standards  for 
measuring  overshoot  and  settling  for  industrial  robots  and  robot  systems. 
ANSI/RIA  15.01-2  92  outlines  the  standards  for  measuring  cornering  overshoot. 

2.7  Compliance 

Static  compliance  is  the  amount  of  elastic  deflection  of  the  Manipulator 
under  a  static  force.  ANSI/RIA  15.01-1  90  outlines  a  method  for  measuring  the 
three  principal  compliance  components  along  a  robot's  base  coordinate  system. 
Both  torsional  compliance  and  coupling  effects  are  excluded. 

Dynamic  values  (frequency,  damping,  amplitude,  phase)  are  also  important 
design  criteria.  Here  modal  analysis  is  a  useful  tool  for  Manipulator  design. 

2.8  Sensitivity 

Specific  sensitivities  important  to  the  Manipulator  are  payload,  direction  of 
approach,  temperature  and  geometry  properties.  All  of  these  factors  influence  the 
accuracy,  repeatability  and  resolution  of  the  Manipulator  system. 
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2.8.1  Pavload 

Payload  refers  to  the  item  handled  alone  and  does  not  include  the  end- 
effector  or  arm,  which  is  considered  part  on  the  Manipulator.  If  the  arm  has 
multiple  end-effector  attachments,  each  attachment  should  be  benchmarked 
separately  because  of  weight  and  functionality  differences.  The  effect  of  payload  is 
complicated  by  the  fact  that  it  depends  not  only  on  the  mass  of  the  payload  but 
also  on  the  mass  distribution  and  orientation. 
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The  inertia  of  the  Manipulator-payload  system  will  have  substantial  impact 
upon  overshoot  and  oscillation.  System  inertia  is  dependent  upon  the 
instantaneous  payload  position  or  the  load  and  end-effector  position  at  anytime 
given  instant.  System  damping  is  critical  to  attaining  targets  with  minimal 
overshoot  and  settling  time. 

Since  loading  affects  speed,  accuracy  and  resolution,  various  loading 
conditions  must  be  benchmarked.  Appleton  and  Williams  (Appleton  and  Williams 
1987)  recommend  testing  at  0,  50  and  100%  of  maximum  payload  and  at  20,  50, 
or  1 00%  of  speed  available. 
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2.8.2  Direction  of  Approach 

One  of  the  hindering  design  points  discovered  by  Glass  (Glass  1984)  on  the 
original  Grove  "Pipe  Manipulator"  was  limiting  movement  parameters.  He 
observed  that  the  pivoting,  booming,  and  telescoping  characteristics  of  the 
Manipulator  limited  its  productivity  and  usefulness.  These  conditions  made  the 
Manipulator  very  sensitive  to  the  direction  of  approach  chosen  by  the  operator. 

Consider  putting  a  round  peg  in  a  round  hole  as  illustrated  in  figure  2.4. 
This  figure  clearly  demonstrates  the  sensitivity  associated  with  direction  of 
approach  of  a  kind  encountered  in  autonomous  Manipulators. 
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Figure  2.5  -  Peg  in  a  Hole  Problem  (Appleton  and  Williams  1987) 
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2.8.3  Temperature 

Hydraulic  systems,  especially  those  used  in  heavy  equipment,  suffer  from 
the  influence  of  heat.  In  hydraulic  equipment  such  as  the  Manipulator,  oil  is 
pressurized  and  released  as  it  circulates  through  the  system.  Heat  is  generated  and 
the  oil  temperature  rises.  Some  of  this  heat  is  gradually  transferred  to  the 
structure  and  joints  and  serves  to  raise  the  temperature  of  the  machine 
components. 

Gradual  heat  build  up  affects  the  feedback  system,  causing  overall 
positional  drift.  Procedures  should  be  implemented  to  reach  a  stable  operating 
temperature  quickly  and  remain  there.  Sufficient  time  must  be  allowed  for  warm 
up  before  testing  commences.  Care  should  be  taken  to  maintain  temperatures 
during  stoppages. 

2.8.4  Geometry  Properties 

Figure  2.5  illustrates  that  geometry  sensitivity  often  plays  a  role  in 
accentuating  or  attenuating  position  errors.  Revolute  joints  such  as  the  one 
depicted  in  the  Figure  2.5  tend  to  be  greater  sources  of  errors  than  prismatic  joints. 
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Figure  2.6  -  Sensitivity  of  Extended  Arm  (Todd  1986) 
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CHAPTER  THREE 
HUMAN  INFLUENCE  ANALYSIS 

3,1  Human  Factors  Engineering 

Human  factors  engineering  is  the  study  of  new  technological  products  and 

the  people  who  make  them  work.  Adams  (Adams  1989)  provides  a  more  formal 

definition: 

The  field  of  human  factors  engineering  uses  scientific  knowledge  about 
human  behavior  in  specifying  the  design  and  use  of  a  human-machine 
system.  The  aim  is  to  improve  system  efficiency  by  minimizing  human 
error. 

The  United  States  government  played  a  key  role  in  the  evolution  of  human  factors 
research.  During  World  War  II,  engineering  systems  became  increasingly  complex 
and  compelled  the  government  to  establish  new  test  centers  at  Wright-Patterson 
and  Brooks  Air  Force  Bases.  More  recently,  human  factors  engineering,  referred 
to  by  some  as  ergonomics,  has  become  even  more  important  with  more  and  more 
utilization  of  state-of-the-art  microprocessor  control  technology. 

Today's  technological  advancements  continue  to  expand  the  performance 
envelope  of  machines  and  machine  systems  at  an  unprecedented  rate. 
Unfortunately,  these  advancements  come  with  a  rather  hefty  price  tag.  Moreover, 
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accompanying  these  high  tech  and  high  cost  components  is  the  disadvantage  of 
complexity.  This  is  important  to  note  since  "the  need  for  human  factors 
engineering  grows  in  direct  relationship  to  complexity  of  the  man-machine 
systems."  (Olex  et  al.  1983). 

A  common  paradox  encountered  in  advancing  technologies  is  the 
development  of  new  systems  that  can  be  exceptionally  difficult  to  use  and  even 
overwhelm  their  human  operators.  Since  the  human  operator  plays  a  vital  role  in 
the  performance  of  the  Manipulator,  human  factors  cannot  be  over  looked. 
System  efficiency  will  remain  as  dependent  upon  the  capabilities  of  the  operator  as 
it  does  upon  the  capabilities  of  the  Manipulator  and  its  control  system. 

Human  factors  engineering  also  considers  the  social  and  motivational 
issues  in  the  analysis,  design,  implementation,  control  and  operation  of  the 
Manipulator.  However,  these  topics  are  beyond  the  scope  of  this  report.  The 
reader  is  referred  to  Graham  (Graham  1991),  Adams  (Adams  1989)  or  Sanders 
(Sanders  et  al.  1993)  for  further  information. 

3.2  Human  Factors  and  the  Manipulator 

Glass  (Glass  1984)  observed  that  the  Manipulator's  eight  control  levers, 
necessary  for  controlling  the  Manipulator's  eight  degrees-of- freedom,  often  left 
the  operator  hesitating  to  determine  proper  maneuvering  sequences  to  optimize 
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performance.  Consequently,  Dupont  had  a  pipe  manipulating  machine  that  was 
very  difficult  to  control  efficiently.  But,  more  importantly,  when  compared  to  the 
less  expensive  "cherry  picker",  the  "Pipe  Manipulator"  was  found  too 
uneconomical  to  operate.  Both  had  similar  operating  costs  but  the  duty  cycle  of 
the  "Pipe  Manipulator"  was  too  low  and  required  a  large  active  site  to  stay  busy. 
When  not  busy  moving  pipe,  the  retention  of  the  crane  function  would  have 
increased  the  "Pipe  Manipulator's"  flexibility  and  usefulness.  Although  Glass 
observed  other  design  factors  that  hindered  performance  of  the  prototype  machine, 
control  has  been  a  primary  focus  for  the  University  of  Texas  Construction 
Automation  Lab. 

Hughes  (Hughes  1990)  chose  the  man-machine  interface  as  his  topic  of 
research  since  better  control  would  correct  "intolerably  slow  and  clumsy 
operation."  He  envisioned  the  Manipulator  as  a  tool  to  be  used  by  a  craftsworker 
without  any  particular  heavy  equipment  training.  He  also  believed  that  new 
controls  would  provide  a  path  for  continued  advancement  to  higher  levels  of 
automation. 

The  possibility  always  exists,  however,  of  designing  an  autonomous 
Manipulator  too  complex  for  the  average  user  to  efficiently  operate.  This,  of 
course,  defeats  the  purpose  of  creating  a  better,  more  powerful  control  system. 
Recent  undertakings  to  improve  the  Manipulator  control  system  include  Hughes' 
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(Hughes  1990)  "ergosticks"  and  more  recently,  Thomas'  (Thomas  1995) 
Dimension  6  Geometry  Ball.  These  efforts  have  focused  on  teleoperated  control, 
were  the  human  operator  guides  the  Manipulator  through  a  joystick  or  similar 
device.  Clearly  the  human  operator  remains  an  integral  part  of  the  control  loop. 

There  are  a  number  of  human  issues  that  need  investigating  in  order  to 
improve  the  human-Manipulator  interface.  These  include,  but  are  not  limited  to, 
the  following: 

•  safety  (always  the  first  priority), 

•  perceptual  limitations  of  the  human  operator  including  visibility  and 
dexterity, 

•  operator  training, 

•  software  interface  design, 

•  robustness  of  design, 

•  discriminability  of  controls  and  control  axes. 

The  anticipated  outcome  of  applying  human  factors  engineering  is  simple. 
When  applied  effectively  it  should  increase  the  compatibility  of  the  human  operator 
with  the  Manipulator  control  system.  This  approach  does  not  increase  the  overall 
capability  of  the  system,  but  makes  it  more  user  friendly.  The  end  result  is  a 
machine  easier  to  operate  for  the  human  user.  Perhaps  then  the  "cherry  picker" 
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will  become  the  obsolete  piece  of  construction  equipment  Glass  (Glass  1984) 
predicted. 

3.3  Human  Factors  Tests  of  the  Manipulator 

Hughes  (Hughes  1990)  performed  a  series  of  simple  tests  to  validate  his 
hypothesis  that  the  Manipulator  will  compete  with  a  "cherry  picker"  in  pipe 
erection  cost.  In  these  tests,  a  set  of  operators  erected  pipe  using  the  Manipulator 
in  a  simulated  plant  environment.  The  experimental  variable  was  task  completion 
time.  His  "ergosticks"  open  loop  rate  interface  time  was  the  test  variable  and  the 
Grove  interface  time  was  the  control.  Other  measurements  consisted  of  all 
operator  inputs  and  Manipulator  main  boom  positions  versus  time. 

The  tests  performed  were  simple  but  effective.  Four  inch  plastic  pipe  was 
picked  up  from  a  lay  down  area  at  ground  level  and  placed  on  an  elevated  pipe 
rack.  Figure  3.1  illustrates  the  test  layout. 

During  the  execution  of  these  tests,  Hughes  made  some  interesting 
discoveries  about  the  human  factors  associated  with  his  new  controller.  He 
explains  in  detail  the  problems  associated  with  operator  vision  during 
teleoperation.  Among  his  findings  was  that  translational  and  angular  alignment  of 
the  Manipulator  jaws  from  a  remote  command  site  had  certain  affixed  human 
limitations.  Accurate  pick  and  placement  of  the  pipe  was  dependent  upon  the 
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operator's  depth  perception  at  a  distance  of  40  feet. 
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Figure  3.1  -  Hughes'  Validation  Test  Arrangement 
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In  addition,  general  observances  regarding  the  "ergosticks"  from  the  test 
operators  summarized  repeated  confusion  between  control  axes,  neutral  positions 
were  not  firm  enough  and  control  arms  were  uncomfortable/wrong  height.  In 
contrast,  the  Grove  controls  seemed  complicated  and  difficult  to  distinguish  during 
lifting  operations.  He  concluded  that  close  attention  must  be  paid  to  the  location 
of  the  controls  and  the  tasks  of  the  Manipulator  operator.  It  is  not  surprising  that 
these  observations  are  associated  with  human  factors. 

Testing  is  important  for  validating  and  quantifying  any  design 
improvement.  Hughes'  tests  were  simple  to  set  up  and  run,  and  represented  a 
typical  task  regularly  performed  on  construction  sites. 

3.4  The  Learning  Curve 

Nof  (Nof,  1985)  describes  learning  "as  the  process  by  which  the  time  or 

the  cost  per  cycle  decreases  as  the  number  of  performed,  repetitive  cycles 

increases."  The  learning  process  has  been  observed  in  humans  to  follow  the 

typical  learning  curve  given  by  the  equation: 

T(/7)  =  T(l)x>?-A 

where:  T  =  time  (or  cost)  per  cycle 

T(l)  =  time  (or  cost)  of  the  first  cycle 

n  =  the  number  of  cycles  performed 

A  =  an  improvement  constant,  determined  by  the  learning  rate. 
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Figure  3.2  shows  a  typical  learning  curve. 
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Figure  3.2  -  The  Learning  Curve  (Nof  1985) 
The  main  factors  in  human  learning  are  (Nof  1985): 

1.  person's  age; 

2.  the  amount  of  previous  experience  in  learning; 

3.  personal  physical  and  psychological  capabilities, 

4.  the  job  complexity  in  terms  of  cycle  length,  amount  of  uncertainty,  and 
degree  of  similarity  to  previous  jobs. 

Now  we  can  describe  mathematically  the  learning  curve  associated  with  each  new 
control  system  tested. 

Hughes'  recorded  the  average  erection  time  versus  the  first,  second  and 
third  pipe  spool  erected  for  each  interface  configuration  (Grove  and  "ergosticks"). 
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Hughes'  Test  Results 

Erection 
time  (sec) 

Pipe  Spool 
Number 

245 

1      "l 

240 
202 

2 
3 

"Ergosticks" 

300 
215 

1      ~~ 
2 

Grove 

225 

3 

The  graph  is  shown  in  Figure  3.3. 
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Figure  3.3  -  Learning  Curve  for  Hughes'  Pipe  Spool  Erection  Test 
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Note  that  smoother  curves  can  be  attained  by  obtaining  more  data  and 
curve  fitting  the  results  as  shown  in  Figure  3.1.  It  is  clear  from  this  graph  that 
both  curves  are  nearly  flat.  This  is  an  indicator  of  the  difficulty  in  operating  each 
system. 

By  performing  the  identical  test  on  new  control  systems,  we  can  directly 
compare  the  curves  (old  versus  new)  and  quantify,  both  numerically  and 
graphically,  the  improvement  associated  with  each  new  control  system.  The 
"steepness"  of  the  learning  curve  will  be  the  indicator  of  the  operator's  ability  to 
learn  the  system.  The  "steeper"  the  curve  the  quicker  it  is  to  learn. 
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CHAPTER  FOUR 
APPLYING  STATISTICAL  ANALYSIS 

Statistics  is  a  science  for  analyzing  data  to  ascertain  errors,  precision,  and 
general  validity  of  experimental  measurements.  One  of  the  major  branches  of 
statistics  is  probability.  Probability  provides  tools  and  methods  for  describing 
random  variations  in  a  system. 

This  chapter  will  examine  the  statistical  methods  for  determining  the 
probability  that  the  Manipulator  will  achieve  a  desired  target  within  a  calculated 
standard  deviation.  The  formulas  for  computing  position  accuracy,  repeatability 
and  their  respective  standard  deviations  will  be  reviewed  for  both  static  and  path- 
related  output.  But  first,  we  shall  briefly  review  the  various  sources  of  error  that 
hinder  performance. 

4.1  Error  Types 

Inherent  in  the  Manipulator  are  various  sources  of  error  that  adversely 
affect  its  accuracy  and  repeatability  performance.  Improving  the  Manipulator's 
performance  will  require  understanding  these  sources.  The  factors  that  influence 
performance  are  categorized  as  geometric,  nongeometric,  or  dynamic  in  nature 
(Hudgens  and  Tesar,  1992).  A  brief  description  of  each  is  given  below: 
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•  Geometric  factors  are  essentially  state  independent  constants  that 
define  the  static  input/output  relationship  between  desired  output 
coordinates  and  Manipulator  achieved  coordinates.  These  error 
sources  include:  kinematic  parameter  error,  Manipulator  placement 
error,  encoder  resolution,  gear  error,  etc. 

•  Nongeometric  factors  are  dependent  upon  load  and  environmental 
conditions  and  affect  the  kinematic  input/output  connection  of  the 
system.  These  errors  include:  compliance,  gravity,  backlash, 
temperature,  etc. 

•  Dynamic  factors  affect  the  higher  order  performance  of  the 
Manipulator  and  are  also  state  dependent.  These  errors  include: 
inertia,  friction,  vibration,  control  system  dynamics,  etc. 

Together  these  sources  of  error  make  it  impossible  for  the  Manipulator  to  achieve 
desired  outputs  exactly. 

The  robotics  community  employs  several  performance  enhancement 
solution  techniques  to  improve  industrial  robot  performance.  These  techniques 
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are  categorized  as  (1)  design,  (2)  sensing,  or  (3)  control  (Hudgen  and  Tesar, 
1992).  Design  solutions  are  based  on  improvements  in  robot  construction  such  as 
employing  higher  manufacturing  tolerances  or  designing  lighter  and  stiffer 
components.  This  is  perhaps  a  long  term  solution  for  future  Manipulators  but  not 
applicable  to  our  current  version.  Sensor  enhancement  solutions  use  an  external 
sensing  system  to  eliminate  error  by  adjusting  joint  feedback  signals.  This  is  very 
expensive  and  not  practical  for  the  Manipulator.  Finally,  the  control  enhancement 
solutions  typically  implement  standard  independent-axis  PID  {Proportional 
integral  Derivative)  control  strategies.  Although  this  solution  is  more 
economical,  the  level  of  sophistication  might  prohibit  its  use  on  the  Manipulator. 

Whether  the  Manipulator  will  need  a  performance  enhancement  boost  will 
depend  upon  the  statistical  data  collected  from  performance  testing  The  next 
section  describes  the  statistical  analysis  presented  in  ANSI/RIA  15.05-2  90  and 
92. 
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4.2  Statistical  Formulas 

The  following  statistical  formulas  have  been  defined  by  ANSI  to  quantify 
the  static  and  path  performance  criteria  of  Robots.  This  report  will  apply  the 
same  performance  criteria  to  the  Manipulator. 

Position  accuracy  is  a  statistical  measure  of  the  spatial  deviation  between 
commanded  and  achieved  Manipulator  positions.  It  is  the  measured  difference 
between  the  commanded  pose  and  the  attained  pose  of  the  Manipulator.  Here 
pose  is  defined  as  a  position  and  orientation  in  space.  The  magnitude  of  the 
accuracy  deviation  (d,)  at  the  /th  position  is  given  by: 

4  =  >/(*., -*JMn,-n,)2+(z..-z«,)2. 

where  Jfai,  ^ai  and  Zai  are  the  coordinates  of  the  attained  pose  at  the  /th 
measurement  and  XCi,  Yei  and  Zcl  are  the  corresponding  commanded  pose 
coordinates. 

4.2.1  Static  Accuracy 

The  two  statistical  measures  for  static  accuracy  are  mean  and  standard 
deviation.  The  mean  position  accuracy  (dpa)  is  given  by: 

1   N 

And  its  standard  deviation  (5,PA)is: 
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2>,-^pa)2 

N-l 

where  N  is  the  total  number  of  measurements.  For  engineering  experiments, 
Holman  (Holman  1984)  maintains  that  it  is  desirable  to  use  at  least  20 
measurements  in  order  to  obtain  reliable  estimates.  ANSI  suggests  at  least  50 
measurements  for  testing  robotic  systems. 

4.2.2  Path  Accuracy 

ANSI  defines  path  accuracy  as  the  measurement  of  the  distance  between  a 
reference  path  and  any  given  attained  path.  Two  types  of  path  accuracy,  relative 
and  absolute,  are  presented  in  ANSI/RIA  15.05-2  92.  They  differ  only  in  the 
definition  of  the  reference  path  used  to  compute  deviations.  However,  ANSI 
recommends  using  the  relative  path  type  because  this  approach  simplifies  the 
necessary  measurement  methods.  Nevertheless,  the  following  formulas  apply 
regardless  of  which  type  is  used.  The  relative  path  accuracy  test,  and  all  other 
tests,  will  be  outlined  in  chapter  five. 

The  two  statistical  measures  for  path  accuracy  are  maximum  and  average 
deviation.  ANSI  defines  maximum  deviation  (AC)  as  the  maximum  distance 
between  any  given  path  and  the  corresponding  reference  path.  The  magnitude  of 
the  maximum  deviation  of  the  worst  path  (ACrel)  is  given  by: 
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ACREL=max  max^U.,  -Ury)2  +(Vt</  - \)2 , 

where  n  is  the  number  of  measurement  cycles  (minimum  of  10),  (Ua  ,  Va  )  are  the 
coordinates  of  the  attained  path  and  (Ur  ,  Vr  )  the  coordinates  of  the  reference 
path  for  the  /th  cycle  andyth  evaluation  point.  See  Figure  4.1. 


Figure  4.1  -  Path  Accuracy  Definition  (ANSI/RIA  R15 .05-2  92) 
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The  average  deviation  is  the  average  of  the  distances  between  any  given 
attained  path  and  the  corresponding  reference  path.  The  magnitude  of  the 
average  relative  path  accuracy  (AC,^)  is  calculated  as  follows: 

1       m         

aCrhL  =-Z^(uay-ur;)2+(va7-vr7)\ 

where  (Ua  ,  Va  )  are  the  coordinates  defined  by: 

Ua   =~2X    and 

va  =-Zva   . 


4.2.3  Static  Repeatability 

ANSI  defines  static  repeatability  as  the  measure  of  deviations  between 
achieved  output  and  the  mean  of  the  output  after  commanding  the  Manipulator  to 
the  same  pose  n  times  from  the  same  direction.  This  test  will  measure  the 
Manipulator's  unidirectional  repeatability.  Omnidirectional  repeatability  is  the 
measure  of  repeatability  when  the  Manipulator  approaches  from  different 
directions. 

Three  measurement  locations  are  used  in  the  test  and  the  arithmetic  mean 
must  be  calculated  at  all  three  locations  before  calculating  the  mean  and  standard 
deviation  of  repeatability.  The  mean  is  given  by: 
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n 

-*"(a,b,c)    ~  ^^(a.b.c), 
1  =  1 

n 

*(a,b,c)   ~  Z-d   *(a,b,c), 
1  =  1 

n 

^(a.b.c)    ~~  Z^^(a,b,c), 
1=1 

Where  (a,b,c),  are  the  three  measurement  locations  for  the  /th  cycle  for  a  total  of 
n  cycles.  ANSI  suggests  completing  five  hundred  cycles  after  system  stabilization 
has  occurred.  Seventy-five  to  one  hundred  cycles  would  be  more  appropriate  for 
the  Manipulator. 

The  deviation  (d)  for  each  measurement  is  determined  by  the  equation: 

"(a,b,c),    —  \C-^(a,b.c),   —  -*(a,b,c)  )     +  '  *  (a.b.c),   —    Va.b.c)  /      +  (Aa.b.c),   —  Aa.b.c)  / 

Thus,  the  total  mean  repeatability  (rf^p)  is: 

n  n  n 


"rep 


i=l  1=1  i=l 


3n 

Now,  the  standard  deviation  (Srep)  can  be  computed  from: 


n  n  n 

H(du   -^REP)2   +  ZK  -^REP)2   +  Z(^c,   -^REp)' 

r*  _    tl     1  =  1 1=J 1=J 
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4.2.4  Path  Repeatability 

ANSI  defines  path  repeatability  as  the  measure  of  the  closeness  between 
multiple  paths.  The  statistical  data  calculated  for  path  repeatability  is  same  as 
relative  path  accuracy  except  that  dynamic  path  repeatability  uses  a  reference  path 
that  is  the  average  of  a  path  traversed  n  times.  ANSI  recommends  completing  a 
minimum  often  (n  =  10)  path  cycles. 

Path  repeatability  is  a  scalar  value  that  represents  the  magnitude  of  the 
deviations  in  a  given  evaluation  plane  (defined  in  Section  5.2.7).  These  deviations 
are  measured  in  the  coordinate  system  of  the  evaluation  plane  (see  Figure  4.2). 
The  path  deviation  (Dl})  is  given  by: 


The  maximum  path  repeatability  (PR)  is: 


m 


PR  =  max  max  D 

j=i       i=i       J 


And  the  average  path  repeatability  (PR)  is: 


PR  =  max  -ZdD{ 


i"     n-    » 


Again,  m  is  the  number  of  evaluation  points  (j),  n  is  the  number  of  measured 
cycles  (I)  and  (Ua  ,  Va  )  are  the  coordinates  defined  by: 
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U     =r2X  and 


1=1 


-        If 

va  =-Zva 

*J      nf^     * 


(Xajj+,  Yajj+,  Zajj+) 


Evaluation 
Plane 


(Xay.,Yaij.,  Zajj.) 


Attained  Path 
Reference  Path 


Figure  4.2  -  Projection  of  Attained  Path  onto  Evaluation  Plane 
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4.3  Probability  Distribution 

Repeatability  is  necessary  for  reliable  replication  of  motion  sequences  that 
are  taught  on-line.  Repeatability  is  a  relative  measure  of  precision.  However,  for 
the  reasons  mentioned  earlier,  the  Manipulator  can  never  be  expected  to  reach  the 
target  point  exactly.  The  Manipulator  actually  achieves  a  range  of  positions  some 
distance  from  the  target.  It  is  reasonable  to  expect  that,  over  numerous  attempts 
to  hit  a  target,  the  actual  output  achieved  will  form  a  known  statistical 
distribution. 

Figure  4.3  illustrates  the  concept  of  probability  distribution.  This  figure 
displays  how  the  probability  of  achieving  a  desired  output  is  distributed  over  a 
distance  x.  Each  value  of  the  ordinate  p(x)  gives  the  probability  that  the  output 
will  succeed  in  achieving  a  random  position  x.  Probability  distributions  are  an 
effective  tool  used  to  describe  repeatability  characteristics. 

The  most  commonly  used  probability  distribution  used  to  interpret 
repeatability  characteristics  is  the  normal  or  Gaussian  distribution.  The  equation 
for  this  distribution  is: 

1 


P(x)  =    -t=  r  e^)2/25  ' 


where  X  is  the  arithmetic  mean  and  S  is  the  standard  deviation  as  previously 
discussed. 


44 


base 

coordinate 

system 


/ 


P(x) 


achieved 
output 


S 


Xm 


Figure  4.3  -  Probability  Distribution  of  Output 
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Figure  4.4  provides  further  illustration  of  the  importance  the  probability 
distribution  has  on  characterizing  Manipulator  output.  Firstly,  the  original  point 
taught  to  the  Manipulator  will  not  necessarily  be  the  arithmetic  mean.  The 
position  of  the  mean  will  be  influenced  by  the  manner  in  which  the  target  was 
originally  taught  and  later  approached.  Consequently,  two  distribution  curves 
result  from  the  two  different  directions  of  approach. 

Secondly,  with  the  use  of  probability  distributions,  it  is  possible  to 
determine  the  likelihood  that  certain  achieved  outputs  will  fall  within  a  specified 
deviation  from  the  mean  of  the  output.  Calculating  the  probability  that  an  output 
will  fall  within  one,  two  and  three  standard  deviations  (S)  of  the  mean  will  result 
in  68.27%,  95.45%  and  99.73%  respectively.  Hence,  it  is  possible  to 
superimpose  positional  tolerances  such  that  99.73%  of  the  output  will  fall  within  a 
specified  band.  Many  robot  suppliers  often  quote  the  width  of  this  band  as  the 
repeatability  of  their  robot  (Appleton  1986). 

And  finally,  the  distance  between  the  LHS  and  RHS  distributions  is  called 
the  mean  hysteresis  range,  and  the  distance  between  three  standard  deviations  is 
called  the  mean  position  variance.  Care  must  be  taken  when  programming  the 
Manipulator  to  ensure  that  taught  points  are  approached  from  the  same  direction 
(unidirectional  repeatability).  Use  of  a  safe,  approximate  stand  off  point  followed 
by  a  slow  approach  to  the  target  will  reduce  the  effects  of  mean  position  variance. 
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Figure  4.4  -  Repeatability  Characteristics  (Appleton  1987) 
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CHAPTER  FIVE 
BENCHMARK  TESTS  FOR  THE  LARGE  SCALE  MANIPULATOR 

The  intent  of  this  chapter  is  to  define  the  benchmark  test  procedures  for 
evaluating  the  performance  criteria  of  Chapter  2.  Methods  of  measuring  the 
performance  criteria  are  not  discussed  in  this  report.  The  reader  is  referred  to 
chapter  eight  of  Knopf  (Knopf  and  Tesar,  1994)  for  information  concerning  robot 
metrology  equipment. 

The  following  performance  tests  are  based  on  the  American  National 
Standard  ANSI/RIA  15.01-1  90  for  Point-to-Point  and  Static  Performance  and 
ANSI/RIA  15.01-2  92  for  Path-Related  and  Dynamic  Performance  evaluations  for 
industrial  robots  and  robot  systems.  The  static  performance  criteria  are:  accuracy, 
repeatability,  cycle  time,  overshoot  and  settling  time.  For  dynamic  performance 
the  criteria  are:  relative  path  accuracy,  path  repeatability,  path  speed 
characteristics  and  cornering  overshoot.  ANSI  feels  that  these  criteria  represent 
the  best  indication  of  the  overall  static  and  path-related  performance  of  industrial 
robots. 
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5.1  Coordinate  Systems 

This  section  establishes  a  common  set  of  coordinate  systems  that  describe 
the  location  of  the  Manipulator,  end-effector  and  test  data. 

5.1.1  World  Coordinate  System 

This  coordinate  system  establishes  a  fixed  frame  of  reference  and  is  usually 
used  to  describe  the  workcell  layout.  This  is  a  Cartesian  coordinate  system  that 
consists  of  three  translational  coordinates  (Xo,Yo,Zo)  and  three  rotational 
coordinates  (Ao,Bo,C0).  The  translational  coordinates  form  a  right-handed 
coordinate  system  with  the  +Zo  direction  collinear  with  but  opposite  in  direction  to 
the  gravity  vector.  The  rotational  coordinates  are  defined  such  that  Ao,  B0  and  Co 
rotate  about  the  Xo,  Y0  and  Z0  axes  respectively.  Note  that  all  rotation 
coordinates  follow  the  right-hand  rule  convention  (see  Figure  5.1). 

5.1.2  Base  Coordinate  System 

The  Base  Coordinate  System  is  used  to  establish  the  location  of  the 
Manipulator  within  the  World  Coordinate  System.  The  Base  Coordinate  System 
defines  the  location  of  the  standard  test  path  and  provides  a  frame  of  reference  for 
recording  test  results.  The  Base  Coordinate  System  is  a  Cartesian  coordinate 
system  that  consists  of  three  translational  coordinates  (Xi,Yi,Zi)  and  three 
rotational  coordinates  (Ai,Bi,Ci)  comparable  to  the  World  Coordinate  System. 
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Figure  5.1  -  Coordinate  Systems 
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The  origin  of  the  Base  Coordinate  System  is  located  at  the  intersection  of 
the  centerline  of  rotation  of  the  first  axis  and  the  floor  of  the  lab.  The  first  axis  is 
the  first  axis  of  motion  encounter  starting  at  the  frame  and  progressing  toward  the 
end-effector.  The  +Xi  axis  points  away  from  the  origin  and  continues  through  the 
centerpoint  of  the  Manipulator  working  space  (Cw)  on  the  plane  defined  by  the 
interface  between  the  Manipulator's  frame  and  the  floor  of  the  lab  (see  Figure  5.1). 

The  working  space  is  the  space  in  which  the  Manipulator  has  no  limitations 
in  the  movement  of  the  mechanical  interface  other  than  those  imposed  by  the 
joints.  The  centerpoint  (Cw)  is  the  geometric  center  of  that  space  (see  Figure  5.2). 
The  +Zi  axis  points  in  the  direction  of  the  mechanical  structure  of  the  Manipulator 
advancing  away  from  the  floor  of  the  lab 

5.1.3  Mechanical  Interface  Coordinate  System 

The  Mechanical  Interface  Coordinate  System  establishes  the  location  of  the 
end-effector  relative  to  the  Manipulator  position.  It  is  a  Cartesian  coordinate 
system  that  consists  of  three  translational  coordinates  (Xm,Ym,Zm)  and  three 
rotational  coordinates  (Am,Bm,Cm)  comparable  to  the  World  and  Base  Coordinate 
Systems. 
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Section   A-A 

Figure    5.2b   -   Reference   Center   Line   E1E2 


The  origin  of  the  Mechanical  Interface  Coordinate  System  is  located  at  the 
intersection  of  the  Manipulator's  Roll,  Pitch  and  Yaw  axes  in  their  mid-positions 
(see  Figures  5. 1  and  5.3).  The  +Zm  axis  is  defined  by  the  centerline  of  the  Roll 
axis  and  points  outwardly  normal  to  the  mechanical  flange  of  the  Manipulator. 
The  +Xm  axis  is  defined  by  the  centerline  of  the  Yaw  axis  and  points  away  from 
the  +Zi  axis.  The  +Ym  direction  is  defined  by  the  right-hand  rule  coordinate 
system  convention. 


5.1.4  Test  Equipment  Coordinate  System 

The  Test  Equipment  Coordinate  System  establishes  the  direction  of  the 
sensor  output.  It  is  a  Cartesian  coordinate  system  that  consists  of  three 
translational  coordinates  (X,,Yt,Z,)  and  three  rotational  coordinates  (At,Bt,Ct) 
comparable  to  the  other  systems.  The  (Xt,Yt,Zt)  coordinates  form  a  right-handed 
coordinate  system  with  the  origin  established  by  the  measurement  apparatus.  The 
vector  relationship  between  the  Base  Coordinate  System  and  the  Test  Equipment 
Coordinate  System  shall  be  recorded  with  the  test  output. 
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5.2  Standard  Test  Conditions 

ANSI  recommends  the  following  conditions  under  which  the  standard  test 
should  be  performed. 


5.2.1  Test  Environment 

The  environment  shall  be  maintained  at  the  following  conditions: 

1.  The  ambient  temperature  shall  be  18°C  to  30°C  and  be  maintained 
within  a  total  range  of  2°C. 

2.  The  relative  humidity  shall  be  maintained  between  30  and  90%. 

3.  The  vibration  content  shall  be  measured  and  noted  if  believed  to 
significantly  affect  the  test  results. 

These  test  conditions  shall  be  noted  with  the  test  results. 

The  intent  of  these  rigid  test  conditions  is  to  achieve  consistent 
performance.  It  is  important  to  ascertain  the  Manipulator's  full  performance 
potential.  Obviously  these  conditions  will  not  be  met  during  outside  crane 
mounted  testing.  Regardless,  the  environment  conditions  shall  always  be  noted. 
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5.2.2  Test  Load 

The  rated  payload  of  the  Manipulator  is  greater  than  140  kg  and  thus  falls 
into  ANSI  Standard  Test  Load  Category  12.  For  this  class,  ANSI  recommends  a 
payload  of  no  less  than  50%  of  the  manufacturer's  maximum  rated  specification. 
Since  the  Manipulator's  maximum  payload  is  currently  1600  lbs.  (725  kg),  the 
standard  test  load  shall  be  no  less  than  800  lbs.  (363  kg).  The  weight,  moment  and 
inertial  properties  of  the  test  load  shall  be  recorded. 


5.2.3  Test  Point 

The  test  point  is  the  physical  point  on  the  end-effector  where  the 
Manipulator  position  is  measured.  Per  ANSI  recommendations,  the  test  point 
shall  be  located  as  close  as  possible  to,  but  not  necessarily  coincident  with,  the 
center  of  gravity  of  the  test  load.  The  axial  and  radial  offset  of  the  test  point  will 
be  greater  than  or  equal  to  the  axial  and  radial  offset  of  the  test  load  center  of 
gravity.  The  following  definitions  apply: 

•     axial  offset.  The  distance  along  the  Zm  axis  of  the  Mechanical  Interface 
Coordinate  System  to  the  center  of  gravity  of  the  test  load. 
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radial  offset.  The  perpendicular  distance  from  the  Zm  axis  of  the 
Mechanical  Interface  Coordinate  System  to  the  center  of  gravity  of  the 
test  load  (see  Figure  5.3). 


5.2.4  Test  Plane 


The  test  plane  is  an  unbounded  referenced  plane  within  the  Manipulator 
working  space  that  is  parallel  to  the  (1,  1,-1)  plane  and  passes  through  the 
working  space  center  point,  Cw  (see  Figure  5.4).  The  test  path  lies  within  the  test 
plane. 


i 
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/ 

Figure  5.4  -  Test  Plane  Location  (ANSI/RIA  Rl 5.05-1  90) 
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5.2.5  Test  Path 

The  test  path  is  a  sequence  of  points  used  to  quantify  the  performance  of 
the  Manipulator.  The  test  path  specified  by  ANSI  allows  for  the  relative 
comparison  of  accuracy,  repeatability,  cyclic  rate  and  overshoot  between  different 
large  scale  manipulators.  All  data  shall  be  taken  from  measurements  of  motion 
along  the  test  path.  The  specific  measurements  that  quantify  the  test  paths  for 
static  and  dynamic  testing  are  described  in  the  following  sections. 


5.2.5.1  Static  (Point  to  Point)  Test  Path 

This  test  path  is  adopted  from  ANSI/RIA  Rl 5.05-1  90.  The  path  is 
located  in  the  test  plane  and  lies  along  the  reference  center  line  EiE2  (see  Figure 
5.2b).  Points  Ei  and  E2  are  located  at  the  intersection  of  the  test  plane  and  the 
boundary  of  the  Manipulator  working  space  on  a  horizontal  line  that  passes 
through  the  working  space  center  point,  Cw     The  test  path  (Figure  5.5)  is  defined 
as: 

1 .    A  rectangle  located  in  the  test  plane  with  Li,  Ui,  U4  and  L4  forming  the 
corners. 
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2.  The  test  path  segment  length  (Sl)  will  be  1000  mm,  the  largest  test 
segment  recommended  by  ANSI.  The  path  will  contain  at  least  three 
segments. 

3.  The  rectangular  segment  side  lengths  (Dl)  will  be  one-half  the  path 
segment  length  (SL)  or  500  mm 

4.  The  total  length  of  the  test  path  rectangle  is  defined  as  the  length  of  the 
line  F1F2  (3000  mm).  The  segment  end-points  are  labeled  Ui  through  U4 
along  the  top  (line  U1U4)  and  Li  through  L4  along  the  bottom  (line  L1L4). 

5.  The  length  of  line  segments  E1F1  and  E2F2  will  be  equal. 

6.  The  Manipulator  shall  maintain,  where  possible,  the  orientation  of  the 
Mechanical  Interface  axis  Zm  perpendicular  to  the  test  plane  at  all  points. 
Where  the  limiting  range  of  the  pivoting  motion  precludes  this  (Glass 
1984),  the  Manipulator  shall  maintain  the  Zm  axis  as  near  to  perpendicular 
to  the  test  plane  as  possible. 


60 


1 

1 

J. 

u2 

u3 

u4 

_' . 

i  i. 

<-- 

F, 

E 

l.UIII 

1.V1I1 

1.U1IJ 

&...> 

5m 

A. 

J. 

I* 

L 

l 

L2 

L3 

L4 

J 

SL 

1 

Figure  5.5  -  Static  Test  Path  (lab  tests) 


5.2.5.2  Dynamic  (Path-Related)  Test  Path 

This  test  path  is  adopted  from  ANSI/RIA  Rl  5.05-2  92    The  path  is 
located  in  the  test  plane  and  lies  along  the  reference  center  line  EiE2  (see  Figure 
5.2b).  Points  Ei  and  E2  are  located  at  the  intersection  of  the  test  plane  and  the 
boundary  of  the  Manipulator  working  space  on  a  horizontal  line  that  passes 
through  the  working  space  center  point,  Cw..  The  test  path  (Figure  5.5)  is  defined 
as: 

1 .    A  rectangle  such  that  the  path  motions  result  in  the  Manipulator 
moving  the  test  load  through  a  large  area  of  the  Manipulator's  working 
space. 
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2.  The  center  of  the  rectangle  will  be  the  midpoint,  Cp,  of  the  center  line 
E,E2 

3.  The  rectangle  will  be  defined  (see  Figure  5.6)  by  the  four  corner  points 
(Ri,  R.2,  R.3,  R4) 

4.  The  segment  lengths,  Sl,  shall  be  1000mm  as  recommended  by  ANSI. 

5.  The  rectangle  will  have  a  height  of  Sl  (1000mm)  and  a  length  of  2Sl 
(2000mm). 

6.  The  direction  of  travel  will  be  clockwise  when  viewed  from  the  base  of 
the  Manipulator. 

7.  The  starting  point  will  be  as  shown  in  Figure  5.6. 

8.  ANSI  recommends  a  maximum  speed  of  lOOOmm/sec  for  an 
SL=  1000mm. 
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R3 

*                           2  SL                                       * 

Figure  5.6  -  Dynamic  Test  Path  (lab  tests) 


5.2.6  Working  Space  Center  Point 

The  working  space  center  point,  Cw,  of  the  Manipulator  is  located  at  the 
midpoint  of  the  line  parallel  to  the  Xi  axis  whose  Zi  axis  position  will  be  the 
midpoint  of  travel  in  the  Z\  axis  direction  (see  Figure  5.2). 


5.2.7  Path-Related  Evaluation  Planes 

ANSI  (ANSI/RIA  Rl  5.05-2  92)  introduces  the  concept  of  evaluation 
planes  to  simplify  the  calculation  process  by  transforming  complex  three- 
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dimensional  path  calculations  into  the  intersections  between  the  attained  path  and 
the  two-dimensional  evaluation  planes  (see  Figure  5.7).  Evaluation  planes  are 
used  to  establish  discrete  locations  for  the  evaluation  of  path  accuracy  and 
repeatability.  These  planes  are  aligned  normal  to  the  test  plane  and  are  placed  at 
equal  linear  distances  of  1/4  Sl  apart  (250  mm).  There  are  a  total  of  20  evaluation 
points.  Linear  interpolation  shall  be  used  when  an  attained  point  does  not  lie 
exactly  on  the  evaluation  plane  (see  Figure  4.2). 
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« »< ► 


< »« >« I  4 ► 


-2  Si 
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Attained  Path     0  Evaluation  Point 


Figure  5.7  -  Path  Related  Evaluation  Planes 
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5.3  Performance  Classes 

ANSI  provides  different  performance  classes  for  optimizing  specific 
performance  characteristics  (for  example,  repeatability  or  cycle  time).  Hardware 
and  software  adjustments  can  be  made  prior  to  testing  for  each  performance  class. 
However,  all  adjustments  must  remain  constant  for  every  test  within  the 
performance  class.  The  four  classes  are: 

•  Class  I-  Standard :  To  evaluate  overall  performance  without  optimizing 
specific  parameters.  The  standard  test  conditions  were  outlined  in  the  previous 
section.  Per  ANSI  guidelines,  the  Manipulator,  a  hydraulic  system,  may  be 
operated  for  15  minutes  prior  to  data  acquisition.  The  Manipulator  shall  not 
experience  system  overloads  or  overheating  during  testing. 

•  Class  II  -  Cycle  Time  (Speed)  :  To  evaluate  the  Manipulator  under  optimized 
cycle  time  conditions.  Performance  parameters  may  be  varied  to  enhance  cycle 
time  performance  consistent  with  intended  use  but  still  remain  indicative  of  actual 
cycle  time  performance. 

•  Class  HI  -  Repeatability:  To  evaluate  the  Manipulator  under  optimized 
repeatability  conditions.  Performance  parameters  may  be  varied  to  enhance 
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repeatability  performance  consistent  with  intended  use  but  still  remain  indicative  of 
actual  repeatability  performance. 

•  Class  IV  -  Special:  To  evaluate  other  specific  Manipulator  performance 
characteristics.  This  class  is  provided  to  allow  testing  of  characteristics  not 
covered  in  classes  I  through  III. 


5.4  Performance  Criteria  for  Point  to  Point  and  Static  Testing 

This  section  outlines  the  static  and  point  to  point  performance  criteria  of 
the  Manipulator.  The  following  subsections  are  derived  from  ANSI/RIA  Rl 5.05-1 
90. 


5.4.1  Positional  Accuracy  (PA) 

ANSI  defines  static  positional  accuracy  as  the  statistical  measure  of  the 
spatial  deviation  between  commanded  and  achieved  Manipulator  positions.  The 
static  accuracy  will  be  computed  from  the  data  collected  during  operation  of  the 
Manipulator  under  the  standard  test  conditions  outlined  in  section  5.2.  The 
recommended  procedure  is: 
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1 .  Calibrate  the  Manipulator  control  system.  Enter  the  coordinates  of  the  test 
point  into  the  controller. 

2.  Match  the  Test  Equipment  Coordinate  System  with  the  Base  Coordinate 
System  at  three  or  more  points  along  the  test  path. 

3.  Enter  the  commanded  poses  into  the  Manipulator  controller  without  physically 
moving  the  Manipulator.  The  list  of  poses  will  consist  of  unique  commanded 
poses  (for  example  Xi,  Yi,  Z\,  Aj,  Bi,  Ci).  The  (Xi,  Yi,  Zi)  coordinates  will  be 
identical  to  the  test  path  vertices  described  in  section  5.2.5. 1 .  The  orientation 
coordinates  (Ai,  Bi,  Ci)  shall  be  selected  at  random  from  the  set  of  all  achievable 
positions.  Fifty  poses  will  be  input  to  the  controller  using  multiple  visits  to  each 
vertex. 

4.  Command  the  Manipulator  to  stop  at  the  selected  commanded  poses.  The 
order  of  visiting  each  vertex  shall  be  completely  random  to  provide  unique 
approach  paths.  After  the  Manipulator  has  reached  stabilization,  measure  the 
achieved  pose  in  the  test  equipment  coordinates  (Xt,  Yt,  Zt). 


5.    Compute  the  mean  (dp3)  and  standard  deviation  (5V a)  accuracy  as  described  in 
Section  4.2.1  and  record  results. 
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5.4.2  Positional  Repeatability 

ANSI  defines  positional  repeatability  as  the  measure  of  deviation  between 
achieved  Manipulator  positions  and  the  mean  of  those  positions  after  ordering  the 
Manipulator  to  the  same  pose  N  times.  The  repeatability  will  be  computed  from 
the  data  collected  during  operation  of  the  Manipulator  under  the  standard  test 
conditions  outlined  in  section  5.2.  The  recommended  procedure  is: 

1.  Three  measurement  positions  are  required  and  will  consist  of  the  Li,  L2,  and  L4 
positions  shown  in  Figure  5.5.  The  motion  between  these  measurement  positions 
will  be  along  the  test  path  while  maintaining  the  orientation  of  the  mechanical 
interface. 

2.  Warm-up  drift  and  the  warm-up  period  are  determined  from  the  number  of 
readings  that  elapse,  after  a  cold  start,  until  the  system  reaches  system 
stabilization.  Warm-up  drift  is  the  positional  difference  between  the  first  position 
after  start-up  and  the  first  position  after  reaching  system  stabilization.  The  warm- 
up  period  is  the  time  this  takes  measured  in  minutes. 

3.  Calculate  and  record  the  Manipulator's  mean  positional  repeatability  (d^) 
and  standard  deviation  (Srep)  as  described  in  Section  4.2.3  for  a  recommended 
sample  size  (TV)  of  500  continuous  cycles. 
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5.4.3  Cycle  Time 

Segment  time  and  traverse  cycle  time  are  the  two  figures  recorded  for 
cycle  time. 

•  Segment  cycle  time  is  the  average  time  required  by  the  Manipulator  to  travel 
through  one  segment  of  the  test  path  and  is  recorded  in  seconds. 

•  Traverse  cycle  time  is  the  average  speed  attained  while  completing  relatively 
large  movements. 

The  recommended  procedure  is: 

1 .  The  test  load  and  test  path  previously  described  in  section  5.2  shall  be  used. 

2.  During  the  segment  cycle  portion  of  the  test,  the  Manipulator  shall  be 
programmed  to  following  test  path: 

Li  ->  Ui  ->  U2  ->  L2  ->  U2  -»  U3  -»  L3  ->  U3  ->  U4  ->  L4. 
The  start  and  end  points  of  each  segment  are  the  points  labeled  L.  The 
Manipulator  shall  reach  system  stabilization  at  each  point  before  continuing. 

3.  Upon  reaching  point  L4,  the  Manipulator  shall  return  to  Li  using  the  following 
return  path:  L4  ->  U4  ->  Ui  -»  Li   This  is  the  path  for  measuring  the  traverse 
speed. 
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4.  The  Manipulator  shall  maintain  the  orientation  of  the  mechanical  interface  axis, 
Zm,  throughout  the  test.  The  maximum  deviation  for  the  programmed  upper  point 
(Ui,  U2,  U3,  U4)  shall  be  recorded. 

5.  Compute  segment  cycle  time  as  follows: 

.               Time(L,  toL4) 
segment  cycle  time  = ! seconds/segment 

where  Time(Li  to  L4)  is  the  time  required  to  travel  from  Li  to  L4  in  step  2. 
Segment  time  is  the  average  number  of  seconds  required  to  move  through  one 
segment.  The  total  number  of  segments  in  this  test  path  is  three. 

6.  Compute  average  traverse  speed  as  follows: 

.      3(SL)  +  2(DL) 

average  traverse  speed  = —  meters/second. 

Time(L4toL,) 

Average  traverse  speed  is  the  average  speed  achieved  during  step  3. 


5.4.4  Overshoot  and  Settling  Time 

Overshoot  is  measured  to  quantify  the  Manipulator  capability  to  make 
smooth  and  accurate  stops.  Overshoot  is  an  important  performance  parameter  for 
operations  involving  large  inertia's,  high  speed  or  frequent  stops.  To  measure 
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overshoot  the  Manipulator  is  run  continuously  from  Ui  — >  Li  ->  Ui  — >  U2  — >  L2 
-»  U2  — >  U3  — >  L3  ->  U3  ->  U4  ->  L4  ->  U4  and  then  straight  to  Ui  (see  Figure 
5.5).  Overshoot  is  equal  to  the  overtravel  distance  at  point  Li  and  is  an  absolute 
value  along  the  direction  of  points  Ui  to  Li  when  the  Manipulator  approaches 
from  point  Ui.  Overshoot  shall  be  measured  for  several  cycles  and  the  average 
recorded 

Settling  time  is  measured  to  quantify  the  Manipulator's  capability  to  stop 
quickly  at  a  target  point.  To  measure  settling  time,  the  Manipulator  is  run  through 
the  same  cycle  as  overshoot.  When  the  Manipulator  approaches  point  Li  from  Ui 
the  position  of  the  test  point  is  continuously  measured  until  system  stabilization 
reached.   Settling  time  is  measured  as  the  elapsed  time  from  the  instance  of  initial 
crossing  into  the  limit  band  until  the  instance  when  the  Manipulator  remains  within 
the  limit  band  (see  Figure  2.4).  Repeat  the  same  procedure  several  times  and 
record  the  average  value. 
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5.5  Performance  Criteria  for  Path-Related  and  Dynamic  Testing 

This  section  outlines  the  path-related  and  dynamic  performance  criteria  of 
the  Manipulator.  The  following  subsections  are  derived  from  ANSI/RIA  Rl  5.05-2 
92. 

5.5.1  Relative  Path  Accuracy 

ANSI  defines  relative  path  accuracy  as  the  measurement  of  the  distance 
between  a  reference  path  and  any  given  attained  path.  The  relative  path  accuracy 
test  utilizes  a  previously  measured  path  as  reference.  The  reference  path  is 
acquired  by  commanding  the  Manipulator  to  follow  the  standard  test  path  at  the 
standard  test  conditions  described  in  Section  5.2.  The  recommended  procedure  is: 

1 .  Calibrate  the  Manipulator  control  system.  Enter  the  coordinates  of  the  test 
point  into  the  controller. 

2.  Move  the  Manipulator  such  that  the  test  point  coincides  with  the  midpoint,  Cp, 
of  the  test  plane.  The  Cp  is  used  as  the  origin  of  the  coordinate  system  defining  the 
reference  path.  The  Test  Equipment  Coordinate  System  is  aligned  with  the  test 
plane  per  Section  10.2.3.3  of  ANSI/RIA  Rl  5.05-2  92. 
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3.  Program  the  Manipulator  to  follow  the  test  path  described  in  Section  5.2  at  a 
speed  of  lOOmm/sec.  Measure  the  (Xi-j,  YrJ?  Zr,)  points  (see  Figure  4. 1)  for  each  of 
the  m  evaluation  points  shown  in  Figure  5.6.  Use  linear  interpolation  to  calculate 
the  intersection  of  the  relative  reference  path  with  the  evaluation  plane  (see  Figure 
4.2).  These  values  will  be  used  as  the  reference  path  to  measure  relative  path 
accuracy. 

4.  Program  the  Manipulator  to  follow  the  standard  test  path  at  50%  and  100%  of 
maximum  speed.  Measure  the  attained  (Xrj.  Yr,,  Zt})  coordinates  for  each 
evaluation  point. 

5.  Calculate  the  maximum  path  accuracy  (ACrel)  and  the  average  path  accuracy 
(ACj^l)  as  described  in  Section  4.2.2  and  record  the  results. 


5.5.2  Path  Repeatability 

ANSI  defines  path  repeatability  as  the  measure  of  the  closeness  between 
multiple  attained  paths.  Path  repeatability  is  measured  using  the  same  test 
procedure  described  in  Section  5.5.1  above.  The  difference  between  the  measured 
path  accuracy  and  path  repeatability  is  that  path  repeatability  uses  the  average  of  a 
path  traversed  n  times  (see  Section  4.2.4).  Calculate  the  maximum  path 
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repeatability  (PR)  and  average  path  repeatability  (PR)  as  described  in  Section 
4.2.4. 


5.5.3  Path  Speed  Characteristics 

ANSI  defines  four  terms  to  quantify  path  speed  characteristics: 

1.  Path  speed  accuracy  (AS)  :  the  difference  between  the  programmed  speed  and 
the  mean  value  of  the  attained  speed  during  //  traversals  along  the  test  path.  AS  is 
expressed  as  a  percentage  of  the  programmed  speed: 


AS(%)  = 


s"sp 

sp 


1    n  _  1 

x  100    where,!  =   —  ^s,,    and    s,  = — ^s 


m  J=1 


Here: 

sp  is  the  programmed  speed, 

Sij  is  the  attained  speed  for  the  /th  cycle  andyth  evaluation  point; 

m  is  the  number  of  evaluation  points; 

n  is  the  number  of  cycles, 

sis  the  mean  speed  for  one  traversal  of  the  test  path. 
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2.  Path  speed  Repeatability  (RS):  the  closeness  of  agreement  of  the  speeds 
attained  for  the  same  programmed  speed.  Using  the  same  procedure  as  path  speed 
accuracy: 


RS(%) 


xlOO. 


3.  Path  speed  fluctuations  (FS):  the  maximum  deviation  in  speed  for  a  single 
traversal  of  an  attained  path  at  constant  speed  conditions.  Using  the  same 
procedure  as  path  speed  accuracy. 


FS  =  max 


1=1 


max(s,.)-  min(s..) 


j=i 


j=i 


4.  Path  acceleration  time  (TS):  the  time  to  move  from  zero  speed  to  programmed 
speed.  The  time  interval  is  measured  from  the  first  detection  of  motion  to  the 
point  where  the  Manipulator  attains  the  average  speed  within  the  tolerance  defined 
by  the  path  speed  fluctuation  (FS). 
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5.5.4  Cornering  Overshoot 

ANSI  defines  corner  overshoot  (CO)  as  the  maximum  deviation  past  the 
target.  It  is  measured  as  the  largest  deviation  outside  the  reference  path  after  the 
Manipulator  has  "passed"  a  corner.  The  value  of  CO  can  be  calculated  for  each  of 
the  three  corners  traversed  in  the  standard  test  path.  The  equation  for  CO  is: 

CO  =  maxV(Xak  -Xrk)2  +(Yak  -Yrk)2  +  (Zak  -Zrk)2. 

Where: 

•  Xak,  Yak,  Zak  are  the  position  coordinates  on  the  attained  path, 

•  Xrk,  Yrk,  Zrk  are  the  coordinates  along  the  reference  path; 

•  k  is  the  subscript  number  for  each  of  the  discrete  data  points  along  the  path  and 
is  dependent  upon  the  test  equipment  sampling  rate  (see  ANSI/RIA  Rl 5.05-2 
92  for  further  explanation). 
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CHAPTER  SEX 
CONCLUSIONS 

This  report  presents  various  methods  of  performance  qualification  for  large 
scale  construction  manipulators  for  the  purpose  of  ascertaining  physical 
specifications.  The  report  defines  the  most  important  static  and  dynamic 
performance  criteria  and  presents  a  method  for  evaluating  them.  These  criteria  are 
accuracy,  repeatability,  overshoot,  settling  time  and  cycle  time. 

A  set  of  meaningful  benchmark  tests  are  presented  to  gauge  the  overall 
static  and  dynamic  performance  of  different  large  scale  manipulators  in  order  to 
provide  a  means  of  relative  comparison.  These  performance  tests  are  based  on  the 
American  National  Standard  ANSI/RIA  15.01-1  90  for  Point-to-Point  and  Static 
Performance  and  ANSI/RIA  15.01-2  92  for  Path-Related  and  Dynamic 
Performance  Evaluations  of  industrial  robots  and  robot  systems. 

Statistical  methods  are  included  for  calculating  the  probability  that  the 
Manipulator  will  achieve  a  desired  target  within  a  calculated  standard  deviation. 
The  formulas  for  computing  position  accuracy,  repeatability  and  their  respective 
standard  deviations  are  reviewed  for  both  static  and  path-related  output. 
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Finally,  the  report  reviewed  the  application  of  human  factors  engineering 
and  presented  a  method  to  quantify  potential  improvements  to  the  human-machine 
interface. 

The  large  scale  manipulator  owned  by  the  University  of  Texas  is  a  very 
adaptable  machine.  Its  eight  degrees-of-freedom,  sixty-five  foot  working  radius 
and  multi-functional  potential  make  it  a  noteworthy  test  bed  for  developing 
enhanced  control  and  performance  testing  strategies.  With  the  application  of  these 
benchmark  tests,  a  means  of  comparing  the  relative  strengths  and  weaknesses  of 
different  controllers  and  manipulators  can  begin. 
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